This Unity asset allows to render PointCloud2 from ROS into Unity.
NOTE: If your ros-sharp version is old, which uses Subscrbier
instead of UnitySubscriber
, you need to git clone this particular commit.
An rgb-d camera or lidar should be running in a ROS-running PC, which is connected to a Unity-running PC by using a LAN cable. For example, you can do so as the following commands, which are actually for my personal purpose.
roslaunch astra_camera astra.launch
roslaunch pcl_ros voxel_grid_filter.launch gui:=false
roslaunch rain_unity ur5_robotiq_unity_real.launch
enableOpenGL.cs
on Main Camera object.PointCloudSubscriber.cs
on RosConnector object (Assuming that you already have ROSConnector). Then, set the name of pointcloud topic according to your ROS PC.PointCloudRenderer.cs
.Subscriber
, and set Point Size
.Offset
in PointCloudRenderer.cs
.-force-opengl
option via command.enableOpenGL.cs
This script loads the openGL and uses glEnable to give us access to the GL_VERTEX_PROGRAM_POINT_SIZE
which allows us to change the size of points rendered in the cloud.
PointCloudRenderer.cs This script takes an array for the colour and and an array for the position of every point. Positions in Unity use a special class called Vector3 which define the x, y, z coords of a position. Colours in Unity are floating point values for RGB (and Alpha) between 0 and 1.
The public float PointSize will allow you to change the size of all rendered points in the cloud, this can be changed before and during runtime.
PointCloudShader.shader This is a custom shader script which implements a float property called _PointSize. You don't need to do anything with this script, PLEASE DO NOT CHANGE ITS NAME. The pointCloud script automatically creates a material and assigns this custom shader to it for rendering the point cloud.
Editor > GraphicsSettingsEditor.cs This is an editor script which ensures the application is using OpenGL. If you attempt to change the graphics API in Project Settings > Player > Other Settings this script will automatically change the API back to OpenGL and warn you that the editor needs to be restarted. You don't need to do anything with this script. The Point Size functionality depends on OpenGL being used, however, if you must use Direct3D or Vulkan then delete this script and good luck...
PointCloud2 has data field where there are the information (e.g. position, intensity, rgb, etc.) of each point. It may look like this.
header:
seq: 1071
stamp:
secs: 1521699326
nsecs: 676390000
frame_id: "velodyne"
height: 1
width: 66811
fields:
-
name: "x"
offset: 0
datatype: 7
count: 1
-
name: "y"
offset: 4
datatype: 7
count: 1
-
name: "z"
offset: 8
datatype: 7
count: 1
-
name: "intensity"
offset: 16
datatype: 7
count: 1
-
name: "ring"
offset: 20
datatype: 4
count: 1
is_bigendian: False
point_step: 32
row_step: 2137952
data: [235, 171, 54, 190, 53, 107, 250, ...
In this example, each point has 32 data field (as indicated by point_step
), where its position is represented by 0
, 4
, and 8
th element (as indicated by offset
). Likewise, the intensity is shown at 16
th element. This offset information is IMPORTANT to extract any necessary information from a Unity script.
For your project, the data field may not look the same. So, I recommend you to investigate your data first by executing, for example,
rostopic echo -p /camera/depth/points > pcl.txt
Then, open the txt file, you will see how the data field look like.