Jupyter ROS: embedding ROS Data Visualization on Jupyter notebooks

.

What we are going to learn

Learn how to embed ROS data visualization inside a Jupyter notebook.

List of resources used in this post

• ROSject with all the code and simulation: http://www.rosject.io/l/d4b0981/
• Jupyter ROS, the tool that allows us to see ROS data in the notebooks: https://github.com/RoboStack/jupyter-ros

Opening the ROSject

In today’s post, we are going to learn how to see ROS Data inside a Jupyter Notebook using the Jupyter ROS package. Let’s start by clicking on the ROSject link (http://www.rosject.io/l/d4b0981/). You should now have a copy of the ROSject.

Now, let’s open the notebook by clicking the Open ROSject button.

Open ROSject – Juypter ROS Demo on ROSDS

When you open the ROSject, a default notebook will be automatically open with the instructions on how to show ROS Data in Jupyter.

The demos are possible thanks to the Jupyter-ROS project developed by Wolf Vollprecht from Quantstack. All credit goes to him and his team.

What follows is the list of ROS enabled notebook demos that work off-the-shelf by using the ROSDS integrated simulations. The original notebook demos were created by Wolf. In The Construct, we only have added the explanations required to understand the code and launch the simulations, so you can have a live demo without having to install and configure your computer.

The ROSject provides everything already installed and configured. Hence, you don’t need to install Jupyter ROS, you only need to learn how to use it… and then use it! (for instructions about how to install on your own computer, please check the author’s repo)

You can freely use all the material from the ROSject to create your own notebooks with ROS.

• ROS 3D Grid: about how to include an interactive 3D grid inside a notebook.
• ROS Robot: about how to add the robot model inside the 3D grid
• ROS Laser Scan: about how to show the robot laser on the 3D grid.
• ROS TEB Demo: about how to add interactive markers to the 3D viewer

Where to find the notebooks inside the rosject

All the notebook files are contained in the ROSject.

In order to see the actual files, use the IDE (top menu, Tools->IDE). Then navigate through the folders up to notebook_ws->notebooks.

Jupyter ROS – List of files in ROSDS

 

How to modify the provided notebooks directly in ROSDS

Use the Jupyter notebook tool to edit the notebooks included in this rosject (or to create new ones).In case you close the notebook, you can re-open it by going to the top menu and selecting Tools->Jupyter Notebook.

Once you have the notebook open, edit at your own will. Check the Jupyter documentation to understand all the possible options.

ROS Laser Scan demo

In the notebooks of the ROSject, we have four demos, but in this post, we are going to show only how to show Laser Scan in the notebook to make things simpler.

In order to get this working, we are going to use the simulation of a Turtlebot.

Remember that you have to have the ROS Bridge running in order for Jupyter ROS to work. If you haven’t started it yet, launch it now.

1. Go to the ROSDS TAB in your browser
2. On the top menu, select Tools->Shell
3. On the opened shell type the following command:

roslaunch ~/notebook_ws/notebooks/launch/bridge.launch --screen

Launching the simulation

Turtlebot in a green area in ROSDS

A new window should appear and load the simulation. Once loaded you should see something similar to this:

Turtlebot opened in a green area in ROSDS

First, Start the demo

First, import the required classes from Jupyter ROS.

Click on the next cell and then press Shift+Enterto activate the Python code.IMPORTANT: the import of such a class can take some time!. You will know that the code is still running because the number on the left of the cell has changed to a * character. Do not move to the next step until the * is gone.

try:
    from sidecar import Sidecar
except:
    pass
    
from jupyros import ros3d

Second, create an instance of the viewer, connect to ROS and get the TF

Click on the next cell and then press Shift+Enterto activate the Python code.

v = ros3d.Viewer()
rc = ros3d.ROSConnection()
tf_client = ros3d.TFClient(ros=rc, fixed_frame='/base_footprint')

Connect to the topic of the laser

Click on the next cell and then press Shift+Enterto activate the Python code.

laser_view = ros3d.LaserScan(topic="/kobuki/laser/scan", ros=rc, tf_client=tf_client)

Fourth, get the 3D grid and the robot model

Click on the next cell and then press Shift+Enterto activate the Python code.

g = ros3d.GridModel()
import os
urdf = ros3d.URDFModel(ros=rc, tf_client=tf_client, path=os.environ.get('JUPYROS_ASSETS_URL', 'http://localhost:3000'))
v.objects = [g, laser_view, urdf]

Fifth, visualize everything

Click on the next cell and then press Shift+Enterto activate the Python code

try:
    sc = Sidecar(title='Sidecar Output')
    with sc:
        display(v)
except:
    display(v)

Sixth, change the visualization style (if required)

Click on the next cell and then press Shift+Enterto activate the Python code.

v.layout.height= '1000px'
g.color = '#CCC'
laser_view.point_size = 0.1
laser_view.static_color = "green"

Move the robot around and watch the laser change!

Open a new shell (Tools -> Shell) and type the following command to make the robot move around:

rostopic pub /cmd_vel geometry_msgs/Twist "linear:
  x: 0.2
  y: 0.0
  z: 0.0
angular:
  x: 0.0
  y: 0.0
  z: 0.3"

That is it. If you did everything ok, you should see the robot moving around in the jupyter notebook.

Youtube video

So this is the post for today. Remember that we have the live version of this post on YouTube. If you liked the content, please consider subscribing to our youtube channel. We are publishing new content ~every day.

Keep pushing your ROS Learning.

Overlaying ROS Workspaces

One of the critical topics of ROS, overlaying workspaces, something that can be confusing even for those who are working with ROS for some time. In this post, we are going to clarify why it happens and how to manage it.

ROSDS Initial environment

As usual, we are going to work with ROSDS, the ROS Development Studio. Creating a new ROSJect, you will have 2 workspaces in your environment, from scratch. One more designed to store ROSDS public simulations and ROS pre-installed packages. Let check this out!

Open a web shell and type the following command:

user:~$ echo $ROS_PACKAGE_PATH

You must see something like:

/home/user/catkin_ws/src:/home/user/simulation_ws/src:/home/simulations/public_sim_ws/src:/opt/ros/kinetic/share

The workspaces are separated by “:”, let’s check one by one, following the order (which is very important!)

1. /home/user/catkin_ws/src
2. /home/user/simulation_ws/src
3. /home/simulations/public_sim_ws/src
4. /opt/ros/kinetic/share

The order was pre-defined by TheConstruct engineers team, to make it suitable for working with public simulations and custom simulations from the user. The order means:

ROS commands are going to look for packages starting from the workspace /home/user/catkin_ws/src, then /home/user/simulation_ws/src and so on. Remember, you can NOT have packages with the same name in the same workspace. But you can have packages with the same name in different workspaces! If the same package exists in two or more workspaces, the first one will be used.

So, if you want to overwrite a simulation from /home/simulations/public_sim_ws/src, you can do just creating/cloning the package with the same name at /home/user/simulation_ws/src.

Re-defining the $ROS_PACKAGE_PATH

“What if I want to add a new workspace to $ROS_PACKAGE_PATH?”

This is an ENVIRONMENT VARIABLE, so is it just a matter of export it the way I want? WRONG!

The ENVIRONMENT VARIABLE is generated by the devel/setup.bash file from each workspace. It means this is just one of the results of sourcing a workspace!

If you need to re-define your $ROS_PACKAGE_PATH, you need to it in a safe/correct way, let’s call it. It is like described below:

1. Source the installation path of ROS:
2. Recompile the workspace you want just after the installation folder:
3. Source its setup.bash:

At this moment, your $ROS_PACKAGE_PATH must be:

/home/user/my_ros_workspace/src:/opt/ros/kinetic/share

You only have 1 workspace defined.

Re-defining the $ROS_PACKAGE_PATH – Practical example

Let’s try it in ROSDS. First, I will create a new workspace:

user:~$ mkdir -p my_ros_workspace/src
user:~$ cd my_ros_workspace/src
user:~/my_ros_workspace/src$ catkin_init_workspace
user:~/my_ros_workspace/src$ cd ..
user:~/my_ros_workspace$ source /opt/ros/kinetic/setup.bash
user:~/my_ros_workspace$ catkin_make

Let’s source the devel/setup.bash of the new workspace and check our $ROS_PACKAGE_PATH:

user:~/my_ros_workspace$ source devel/setup.bash
user:~/my_ros_workspace$ echo $ROS_PACKAGE_PATH

And the result must be:

/home/user/my_ros_workspace/src:/opt/ros/kinetic/share

Even though we have many other workspaces defined, our $ROS_PACKAGE_PATH considers only one workspace! That’s because we have sourced just this workspace’s devel/setup.bash.

Overlaying workspaces

Now, let’s do some more advanced. We want to have more workspaces in our $ROS_PACKAGE_PATH. But let’s check something before:

Execute the following:

user:~$ source catkin_ws/devel/setup.bash
user:~$ echo $ROS_PACKAGE_PATH

You must see:

/home/user/catkin_ws/src:/home/user/simulation_ws/src:/home/simulations/public_sim_ws/src:/opt/ros/kinetic/share

You still have the previous $ROS_PACKAGE_PATH defined, but only for the catkin_ws. The conclusion is each workspace has its own $ROS_PACKAGE_PATH defined.

Now, we are going to add public_sim_ws to our new workspace. This is the way to make the public simulations provided by TheConstruct available for your new workspace.

user:~$ source /home/simulations/public_sim_ws/devel/setup.bash
user:~$ echo $ROS_PACKAGE_PATH

You must have:

/home/simulations/public_sim_ws/src:/opt/ros/kinetic/share

It means ROS is only considering the public_sim_ws. Let’s put our new workspace on top of it.

user:~$ cd my_ros_workspace/
user:~/my_ros_workspace$ rm -rf build devel
user:~/my_ros_workspace$ catkin_make
user:~/my_ros_workspace$ source devel/setup.bash

And you must have a completely new $ROS_PACKAGE_PATH

/home/user/my_ros_workspace/src:/home/simulations/public_sim_ws/src:/opt/ros/kinetic/share

Conclusion

This is how you can work with multiple workspaces. Bare in mind ROS creates rules to give priority, but you are the one in charge of configuring the order of your workspaces!

Don’t forget to check the official reference: http://wiki.ros.org/catkin/Tutorials/workspace_overlaying

And if you like this kind of post, don’t forget to share it.

Leave a comment for us to know your personal feedback!

Cheers!

ROS Mini Challenge #9 – Fusing data to improve robot localization with ROS

…

What we are going to learn

Learn how to improve robot localization with data from different sensors

List of resources used in this post

• The ROSject of the project with the simulation and the ROS Packages: buff.ly/2RifSjn
• Robot Ignite Academy courses: www.robotigniteacademy.com/en/

Where to find the code

Once you open the ROSject (buff.ly/2RifSjn), you will get a copy of it. You just have to click Open. Once open, inside the catkin_ws folder, you will find all the ROS packages associated with this challenge. You can have a look at it using the IDE, for instance.

Code Editor (IDE) – ROSDS

Launching the simulation

Choose lunch file to open simulation in ROSDS

 

Now, from the launch files list, select the launch file named rotw9.launch from the sumit_xl_course_basics package.

ROS Mini Challenge #9 – Select launch file

Finally, click on the Launch button to start the simulation. In the end, you should get something like this:

Summit XL gazebo simulation in ROSDS, ROSject of the week

The problem to be solved

As you can see, this ROSject contains 1 package inside its catkin_ws workspace: rotw9_pkg. This package contains a launch file and a configuration file, which are used to start an ekf localization node. This node is provided by the robot_localization package, and its main purpose is to fuse different sensor data inputs in order to improve the localization of a robot.

In our case, we are going to fuse Odometry data (which has been tweaked) with Imu data (which is correct). So, the main purpose of this challenge is to improve the Summit XL odometry data, since the initial one is a little bit distorted.

In order to test that it works as expected, first of all, we need to know the odometry topics. For this, we can use the following command:

rostopic list | grep odom

This command will give us the following topics:

/noisy_odom
/robotnik_base_control/odom

For this challenge, we are going to use the /noisy_odom topic, which basically is the original odometry data with some extra noise added to it.

You can have a look at the current odometry using RViz (rosrun rviz rviz ) and adding an Odometry display. You should get something similar to this:

Noisy Odometry example in ROSDS

As you can see, the Odometry readings are not very stable.

So now let’s start our node in order to correct the odometry readings with the following command:

roslaunch rotw9_pkg start_ekf_localization.launch

This command will generate a new Odometry topic, named /odometry/filtered, which will contain the resulting Odometry data (fusing the /noisy_odom data with the Imu data).

If you visualize this new Odometry, you will get something like this. As you can see, the new Odometry is much more stable than the original one. Great!

Correct (expected) Odometry data in ROSDS

NOTE: In order to properly visualize the differences between the 2 Odometry readings, you should modify the arrow size and color on the different displays.

Solving the ROS Mini Challenge

<launch>

<!-- Run the EKF Localization node -->
<node pkg="robot_localization" type="ekf_localization_node" name="ekf_localization">
<rosparam command="load" file="$(find rotw9_pkg)/config/ekf_localization.yaml"/>
</node>

</launch>

#Configuation for robot odometry EKF
#
frequency: 50

two_d_mode: true

publish_tf: false

odom_frame: odom
base_link_frame: base_link
world_frame: odom

odom0: /noisy_odom
odom0_config: [true, true, false,
               false, false, true,
               true, true, false,
               false, false, true,
               false, false, false]
odom0_differential: false

imu0: /imu/data
imu0_config: [false, false, false,
              false, false, true,
              false, false, false,
              false, false, true,
              true, false, false]
imu0_differential: false

process_noise_covariance": [0.05, 0,    0,    0,    0,    0,    0,     0,     0,    0,    0,    0,    0,    0,    0,
                                              0,    0.05, 0,    0,    0,    0,    0,     0,     0,    0,    0,    0,    0,    0,    0,
                                              0,    0,    0.06, 0,    0,    0,    0,     0,     0,    0,    0,    0,    0,    0,    0,
                                              0,    0,    0,    0.03, 0,    0,    0,     0,     0,    0,    0,    0,    0,    0,    0,
                                              0,    0,    0,    0,    0.03, 0,    0,     0,     0,    0,    0,    0,    0,    0,    0,
                                              0,    0,    0,    0,    0,    0.06, 0,     0,     0,    0,    0,    0,    0,    0,    0,
                                              0,    0,    0,    0,    0,    0,    0.025, 0,     0,    0,    0,    0,    0,    0,    0,
                                              0,    0,    0,    0,    0,    0,    0,     0.025, 0,    0,    0,    0,    0,    0,    0,
                                              0,    0,    0,    0,    0,    0,    0,     0,     0.04, 0,    0,    0,    0,    0,    0,
                                              0,    0,    0,    0,    0,    0,    0,     0,     0,    0.01, 0,    0,    0,    0,    0,
                                              0,    0,    0,    0,    0,    0,    0,     0,     0,    0,    0.01, 0,    0,    0,    0,
                                              0,    0,    0,    0,    0,    0,    0,     0,     0,    0,    0,    0.02, 0,    0,    0,
                                              0,    0,    0,    0,    0,    0,    0,     0,     0,    0,    0,    0,    0.01, 0,    0,
                                              0,    0,    0,    0,    0,    0,    0,     0,     0,    0,    0,    0,    0,    0.01, 0,
                                              0,    0,    0,    0,    0,    0,    0,     0,     0,    0,    0,    0,    0,    0,    0.015]


initial_estimate_covariance: [1e-9, 0,    0,    0,    0,    0,    0,    0,    0,    0,     0,     0,     0,    0,    0,
                                                      0,    1e-9, 0,    0,    0,    0,    0,    0,    0,    0,     0,     0,     0,    0,    0,
                                                      0,    0,    1e-9, 0,    0,    0,    0,    0,    0,    0,     0,     0,     0,    0,    0,
                                                      0,    0,    0,    1e-9, 0,    0,    0,    0,    0,    0,     0,     0,     0,    0,    0,
                                                      0,    0,    0,    0,    1e-9, 0,    0,    0,    0,    0,     0,     0,     0,    0,    0,
                                                      0,    0,    0,    0,    0,    1e-9, 0,    0,    0,    0,     0,     0,     0,    0,    0,
                                                      0,    0,    0,    0,    0,    0,    1e-9, 0,    0,    0,     0,     0,     0,    0,    0,
                                                      0,    0,    0,    0,    0,    0,    0,    1e-9, 0,    0,     0,     0,     0,    0,    0,
                                                      0,    0,    0,    0,    0,    0,    0,    0,    1e-9, 0,     0,     0,     0,    0,    0,
                                                      0,    0,    0,    0,    0,    0,    0,    0,    0,    1e-9,  0,     0,     0,    0,    0,
                                                      0,    0,    0,    0,    0,    0,    0,    0,    0,    0,     1e-9,  0,     0,    0,    0,
                                                      0,    0,    0,    0,    0,    0,    0,    0,    0,    0,     0,     1e-9,  0,    0,    0,
                                                      0,    0,    0,    0,    0,    0,    0,    0,    0,    0,     0,     0,     1e-9, 0,    0,
                                                      0,    0,    0,    0,    0,    0,    0,    0,    0,    0,     0,     0,     0,    1e-9, 0,
                                                      0,    0,    0,    0,    0,    0,    0,    0,    0,    0,     0,     0,     0,    0,    1e-9]

In the YAML file above, the parameters we are going to focus on are:

odom_frame: odom
base_link_frame: base_link
world_frame: odom

We see that the are links named odom and base_link. We have to make sure these links exist in the simulation. Let’s do that by generating a TF Tree with the commands below:

cd ~/catkin_ws/src
rosrun tf view_frames

The commands above generate a file named frames.pdf that contains the full TF Tree of the simulation, with all the connections, the frame names, etc.

You can easily download the frames.pdf using the IDE (Code Editor) by selecting it and clicking Download.

After opening that file, we can see the names of the links:

TF Tree for Summit XL Simulation

As we can see, the Odom link is actually named summit_xl_a_odom, and the base_link is named summit_xl_a_base_link. Let’s then fix the Yaml file with the correct values and save the file:

odom_frame:summit_xl_a_odom
base_link_frame:summit_xl_a_base_link
world_frame:summit_xl_a_odom

Now we can try to launch our localization node again:

roslaunch rotw9_pkg start_ekf_localization.launch

Unfortunately, the red arrow in RViz (topic /odometry/filtered) is still not as we expect.

Let’s now check the odom0_config settings in the YAML file:

odom0: /noisy_odom
odom0_config: [true, true, false,
               false, false, true,
               true, true, false,
               false, false, true,
               false, false, false]
odom0_differential: false

The error is actually in that matrix.

The first row goes for X, Y and X positions. The second for ROW, PITCH and YAW. The third is for the linear values, the fourth the angular velocities and the final row is for the linear acceleration.

The problem is basically because we are inputting some noise in the X and Y positions. More details on this can be found in the documentation of the robot_localization package. If we just set those values to false, we have the following:

odom0: /noisy_odom
odom0_config: [false, false, false,
               false, false, true,
               true, true, false,
               false, false, true,
               false, false, false]
odom0_differential: false

If we now try to launch our localization node again, we should see the correct values in RViz:

roslaunch rotw9_pkg start_ekf_localization.launch

Remember that if you need a deeper understanding on fusing sensor data, the Fuse Sensor Data to Improve Localization course in Robot Ignite Academy (www.robotigniteacademy.com/en/) can help you.

Youtube video

So this is the post for today. Remember that we have the live version of this post on YouTube. If you liked the content, please consider subscribing to our youtube channel. We are publishing new content ~every day.

Keep pushing your ROS Learning.

ROS Mini Challenge #8 – Extracting the highest value from laser readings

What we are going to learn

Learn how to read data from a laser topic and extract the highest value.

List of resources used in this post

• The ROSject of the project with the simulation and the ROS Packages: buff.ly/2PU4ucB
• Robot Ignite Academy courses: www.robotigniteacademy.com/en/

Where to find the code

Once you open the ROSject (http://buff.ly/2PU4ucB), you will get a copy of it. You just have to click Open. Once open, inside the catkin_ws folder, you will find all the ROS packages associated with this challenge. You can have a look at it using the IDE, for instance.

To see the full code, open the IDE by going to the top menu and select Tools->IDE

Code Editor (IDE) – ROSDS

Launching the simulation

Go to the top menu and select Simulations. On the menu that appears, click on the Choose launch file… button.

Choose launch file to open simulation in ROSDS

Now, from the launch files list, select the launch file named rotw5.launch from the rosbot_gazebo package.

Finally, click on the Launch button to start the simulation. In the end, you should get something like this:

Turtlebot robot inside a wall in ROSDS

source ~/catkin_ws/devel/setup.bash

rosrun rotw8_pkg rotw8_code.py

ROS Mini Challenge #8 – Expected output

NOTE: The numbers in the value of the maxim and the position in the array may vary a little bit from the ones in the image above (but not much).

Solving the ROS Mini Challenge

Ok, so… where’s the problem? If you have tried to reproduce the steps described above you have already seen that it DOES NOT WORK. When you run the programs introduced above, the values you get are not the correct ones at all. So… what’s going on?

When you launch rosrun rotw8_pkg rotw8_code.py, the output is not the expected one:

$ rosrun rotw8_pkg rotw8_code.py
Highest value is 4.50035715103 and it is in the position 0 of the array.

The difference is almost 2 meters, and position 0 (zero) is really different from 202, which is the more or less the expected one. The reason why we expect something like position 202 is because, on the right of the robot we have the position 0 in the scan, in the left, we have position 719, right in front we have 360, so the highest distance should be close to the exit, which is around the position 202. You may understand better by looking at the image below:

Laser range positions in the turtlebot simulation in ROSDS

If we check the ~/catkin_ws/src/rotw8_pkg/src/rotw8_code.py file, its original code is:

#! /usr/bin/env python

import rospy
import time
from sensor_msgs.msg import LaserScan

class Challenge:
    def __init__(self):
        self.sub = rospy.Subscriber("/kobuki/laser/scan", LaserScan, self.laser_callback)
        self.laser_msg = LaserScan()
    def laser_callback(self, msg):
        self.laser_msg = msg

    def get_laser_full(self):
        time.sleep(1)
        return [self.laser_msg.ranges[0], self.laser_msg.ranges[719]]

    def get_highest_lowest(self):
        l = self.get_laser_full()
        i = 0
        maxim = -1
        for value in l:
            if value >= maxim and str(value) != "inf":
                maxim = value
                max_pos = i
            i = i + 1

        print "Highest value is " + str(maxim) + " and it is in the position " + str(max_pos) + " of the array."

if __name__ == '__main__':
    rospy.init_node('rotw8_node', anonymous=True)
    challenge_object = Challenge()
    try:
        challenge_object.get_highest_lowest()

    except rospy.ROSInterruptException:
        pass

If we look carefully, we can notice that the problem is in the get_laser_full function

 def get_laser_full(self):
        time.sleep(1)
        return [self.laser_msg.ranges[0], self.laser_msg.ranges[719]]

As we can see, it only returns two values of the laser.

To solve the problem we just change this function by:

 def get_laser_full(self):
        time.sleep(1)
        return self.laser_msg.ranges

If you now save the file and run the “rosrun rotw8_pkg rotw8_code.py” command again, now we should get the desired output:

$ rosrun rotw8_pkg rotw8_code.py
Highest value is 7.23973226547 and it is in the position 203 ofthe array.

Hey, now the value is the correct one. The problem was really easy, wasn’t it?

If you are still struggling to understand the code, or you want to master your ROS Skills, I highly recommend you take some courses in Robot Ignite Academy: http://www.robotigniteacademy.com/en/

Youtube video

So this is the post for today. Remember that we have the live version of this post on YouTube. If you liked the content, please consider subscribing to our youtube channel. We are publishing new content ~every day.

Keep pushing your ROS Learning.

Developing Web Interfaces For ROS Robots #4 – Streaming robot’s camera on the web page

Hello ROS Developers,

This is the 4th of 4 posts of the series Developing web interfaces for ROS Robots.

In this post, we are going to stream the images of the robot’s camera on the webpage.

1 – Loading a new JavaScript library

Before we go to the camera streaming server or the code to open its connection, we need to import the library that is in charge of making everything possible. That is the MJpeg library.

Let’s import it in our <head> section:

    ...

    <script src="https://cdn.jsdelivr.net/npm/eventemitter2@5.0.1/lib/eventemitter2.min.js">
    </script>
    <script type="text/javascript" src="https://static.robotwebtools.org/mjpegcanvasjs/current/mjpegcanvas.min.js">
    </script>

</head>

2 – Adding the camera viewer element to the page

Now for the webpage, we need a new element there which will define where the camera images are going to be shown:

Let’s use the div row we have created for the buttons (in the previous post) to embed the camera. It must be replaced by the following code:

        <div class="row">
            <div class="col-md-6 row">
                <div class="col-md-12 text-center">
                    <h5>Commands</h5>
                </div>

                <!-- 1st row -->
                <div class="col-md-12 text-center">
                    <button @click="forward" :disabled="loading || !connected" class="btn btn-primary">Go straight</button>
                    <br><br>
                </div>

                <!-- 2nd row -->
                <div class="col-md-4 text-center">
                    <button @click="turnLeft" :disabled="loading || !connected" class="btn btn-primary">Turn left</button>
                </div>
                <div class="col-md-4 text-center">
                    <br>
                    <button @click="stop" :disabled="loading || !connected" class="btn btn-danger">Stop</button>
                    <br>
                    <br>
                </div>
                <div class="col-md-4 text-center">
                    <button @click="turnRight" :disabled="loading || !connected" class="btn btn-primary">Turn right</button>
                </div>

                <!-- 3rd row -->
                <div class="col-md-12 text-center">
                    <button @click="backward" :disabled="loading || !connected" class="btn btn-primary">Go straight</button>
                </div>
            </div>

            <div class="col-md-6">
                <div id="mjpeg"></div>
            </div>
        </div>

Yes, the camera code is only the final part:

    ...
    <div class="col-md-6">
        <div id="mjpeg"></div>
    </div>
</div>

3 – Webserver for image streaming

The same way we need start rosbridge server, we need to start a specific ros process to stream the images:

rosrun web_video_server web_video_server _port:=11315

We are specifying the port at the end because of ROSDS, it provides us a specific address to this port. If you are working local, you can access it at locahost:<port>. The port number is shown in the terminal you run it.

4 – Adjusting JavaScript code

In our main.js file, let’s add the camera connector. It’s basically a way to make the pre-defined element (in our HTML) to receive the images.

• Create a new method:

        setCamera: function() {
            console.log('set camera method')
            this.cameraViewer = new MJPEGCANVAS.Viewer({
                divID: 'mjpeg',
                host: '54.167.21.209',
                width: 640,
                height: 480,
                topic: '/camera/rgb/image_raw',
                port: 11315,
            })
        },

Notice that you have to change the attributes host, topic and port accordling to your configuration.

The attributes width and height are going to be applied to the size of the element.

• Call the method whenever the rosbridge server is connected (inside the callback)

            this.ros.on('connection', () => {
                this.logs.unshift((new Date()).toTimeString() + ' - Connected!')
                this.connected = true
                this.loading = false
                this.setCamera()
            })

Pay attention to the IP we are using in the code! You must have a different IP than the one used in this post. If you are using ROSDS, get yout public IP executing the following command:

public_ip

If you are on your local computer, it must be:

localhost

5 – Testing the webpage

At this point you must have something similar to the image below:

We have used a different world this time to have some obstacles that we can identify in the camera. You can switch between many other simulations to see different results on the camera!

ROSJect created along with the post:

http://www.rosject.io/l/ddd21cc/